1. Field of the Invention
The present invention relates to a reproduction control method about a multimedia stream and an apparatus thereof, in particular to a non-linear reproduction control method of a multimedia stream and an apparatus thereof including a forward and a backward skip functions using structural information or semantic information of multimedia content.
2. Description of the Prior Art
Recently, according to development of a digital video technology and an image or a video processing technology, users can search and browse a request part of request multimedia content (drama, movie, documentary etc.) at a request time point.
A shot segmentation technology and a shot clustering technology are basic technologies for performing a non-linear multimedia browsing and searching.
The shot segmentation is a technology for dividing the video stream into each separated shot.
The shot clustering technology is for constructing multimedia content by binding the each shot with logical scenes on the basis of time/image/motion/audio/semantic similarity.
The shot means a sequence of video frames taken from a camera without interruption, it is the most basic unit for analyzing or constructing video, and the video is constructed with lots of shots.
It is possible to browse only intersected section in the video content with the help of structural or semantic index of a multimedia content.
Instance replay function is the most frequently used in non-linear multimedia content browsing.
When a user watches or listens a video or an audio program, the user can watch or listen again an interesting part or a part broadcast for absence by using the instant replay function, the instant replay function can be used in both analog and digital environment.
Recently, it is possible to provide the instant replay function not only about video/audio recorded with an offline but also about a program broadcast or serviced in real-time by the development of a set-top box (Ex Tivo, replay TV) which is capable of performing recording and reproduction concurrently.
Generally, methods for providing the instant replay function uses a set temporal offset using average user reaction time.
When the instant replay is requested, in the method using the average user reaction time, reproduction resumes from a time point which is about 7˜seconds previous to the present position by using the average time (7˜8 seconds) which is the average program miss time of the user in audition. In order to reproduce the video program from more previous scene, the user can reproduce the request part by performing the instant replay function a plurality of times. Because it is difficult to reproduce the exact starting position of the segment what the user want by using only the conventional instant replay function, a rewind function is usually used together.
In other words, in the conventional instant replay using the average user reaction time, the reproduction is performed by resume reproduction from the point which is fixed times before from the present reproduction. Accordingly the method can not be used when the user requests the instant replay based on the structural or semantic information of a multimedia content.
There is methods similar to the instant replay function using the preset temporal offset such as a method using temporal data blank, a method providing an instant replay function by designation of a user, and a method using physical storing position information of a stream.
The method using the temporal data blank is used in most of music programs, when a plurality of songs are recorded on a magnetic tape, transferring to the previous song or after song is performed by searching a blank region of the temporal data by using a characteristic as a certain time interval of data blank which exists between one song and the other song.
However, the method using the temporal data blank can not be used for searching inner part of a stream in data having no temporal blank such as the multimedia stream, it can be used only for searching a limited unit stored in a limited media such as a magnetic tape for music. In addition, because its control unit is very big, it is difficult to use the method for a non-linear reproduction control function required by the user in the general video or audio program. In other words, the method using the temporal data blank is not the reproduction control method for reproducing from the inner side of the video stream but the reproduction control method for reproducing by the stream unit.
The instant replay function supply method by the user designation is a method designating a reproduction start point before the user requests the instant replay, and is mainly used for listening a certain part repeatedly inside of a stream such as a tape for language study.
Because the reproduction start point has to be designated in the method for providing the instant replay function by the user designation, it is very difficult to use the method for generic video or audio program.
The method using the physical storing position information of the video stream is a method used for a digital media such as a disk for music, because the physical start position about each song is recorded on header information of the disk, it can react to a song unit navigation of the user.
The method using the physical storing position information of the video stream is capable of performing easily the searching by the video stream unit, however the instant replay control unit is very big and it can not correspond to a reproduction control request by regions inside of the video stream, accordingly it is difficult to use the method for the reproduction control method about the generic video or audio program, and the method only can provide a reproduction control function by the stream unit like the method using the temporal data blank.